Structured Parameter Estimation for LFG-DOP using Backoff

Despite its state-of-the-art performance, the Data Oriented Parsing (DOP) model has been shown to suffer from biased parameter estimation, and the good performance seems more the result of ad hoc adjustments than correct probabilistic generalization over the data. In recent work, we developed a new estimation procedure, called Backoff Estimation, for DOP models that are based on Phrase-Structure annotations (so called Tree-DOP models). Backoff Estimation deviates from earlier methods in that it treats the model parameters as a highly structured space of correlated events (backoffs), rather than a set of disjoint events. In this paper we show that the problem of biased estimates also holds for DOP models that are based on Lexical-Functional Grammar annotations (i.e. LFG-DOP), and that the LFG-DOP parameters also constitute a hierarchically structured space. Subsequently, we adapt the Backoff Estimation algorithm from Tree-DOP to LFG-DOP models. Backoff Estimation turns out to be a natural solution to some of the specific problems of robust parsing under LFGDOP. The DOP model (Bod, 1995; Bod, 2001) currently exhibits good performance on current benchmark treebanks, e.g. (Marcus et al., 1993). These treebanks are annotated with impoverished phrase-structure parsetrees and the DOP model that fits these annotations, called Tree-DOP, works with the common rewrite operation of substitution ( inherited from Context-Free Grammars). Despite the simplicity of the rewrite formalism underlying Tree-DOP, probability estimation turns out not to be as straightforward as it initially seemed. Earlier studies (Johnson, 2002; Sima’an and Buratto, 2003) have shown the bias of the three previous estimation procedures for Tree-DOP, namely (Bod, 1995), (Bonnema et al., 1999) and (Bod, 2001). Recently, a new study (Sima’an and Buratto, 2003) shows that the main problem with estimation for TreeDOP is that the various parameters cannot be assumed to be disjoint, as earlier work does; in fact, the TreeDOP parameter space is shown to abide by a partial order that structures these parameters according to correlations of occurrence between the different parameters. A suitable algorithm, Backoff Estimation, takes into account this fact during the parameter estimation for Tree-DOP. Preliminary experiments show improved performance, despite the impoverished first implementation. In this paper, we study parameter estimation for the extension of DOP to linguistic annotations that are richer than phrase-structure. We concentrate on the extension of DOP to Lexical-Functional Grammar (LFG) annotations, i.e. LFG-DOP (Bod and Kaplan, 2003). Naturally, the problem of biased parameter estimation carries over from Tree-DOP to LFGDOP. In fact, the bias in Tree-DOP is further compounded with specific aspects of LFG-DOP that allow for robust processing that can be achieved by abstraction over actually occurring treebank structures. We show how the Backoff Estimation procedure applies to LFG-DOP and discuss the resulting model. It turns out that Backoff Estimation naturally realizes the specific model architecture that has been observed to work best in previous experiments with LFG-DOP (Bod and Kaplan, 1998; Bod and Kaplan, 2003). Section 1 provides a review of the Tree-DOP model. Section 2 reviews the Backoff Estimation algorithm. Section 3 reviews LFG-DOP and section 4 extends the Backoff Estimation algorithm to LFGDOP. Finally, section 5 provides the conclusions from this work. 1 Tree-DOP: Phrase-Structure Like other treebank models, Tree-DOP extracts a finite set of rewrite productions, called subtrees, from the training treebank together with probabilities. A connected subgraph of a treebank tree t is called a subtree iff it consists of one or more context-free productions1 from t. Following (Bod, 1995), the set of rewrite productions of Tree-DOP consists of all the subtrees of the treebank trees. Figure 3 exemplifies the set of subtrees extracted from the treebank of Figure 1. Note that a non-leaf node labeled p in tree t dominating a sequence of nodes labeled c1, · · · , cn consists of a graph that represents the context-free production: p → c1 · · · cn.